Individual fields from video sources generally exhibit the following shortcomings:
sensor, tape and transmission noise; PA1 luminance aliasing due to insufficiently dense spatial sampling of the optical scene; PA1 chrominance aliasing due to insufficiently dense spatial sampling of particular color components in the optical scene (often occurs with single CCD video cameras which can only sense one color component at each pixel position); PA1 relatively poor resolution.
However, video sources have the advantage that many pictures of the same scene are available, usually with relatively small displacements of the scene elements between consecutive fields. After suitable compensation for motion, these multiple pictures can be combined to produce a still image with less noise. Perhaps more importantly, however, the existence of motion allows for effectively having a denser sampling of the optical scene than is available from any single field. This opens up the possibility for aliasing removal as well as resolution enhancement.
While analog video is considered, many of the following observations also apply to a variety of digital video sources. One observation is that the resolution of the chrominance components is significantly lower than that of the luminance components. Specifically, the horizontal chrominance resolution of an NTSC (National Television System Standard) broadcast video source is about 1/7 that of the luminance. Also, although the NTSC standard does not limit the vertical resolution of the chrominance components below that of the luminance components, most popular video cameras inherently halve the vertical chrominance resolution, due to their single CCD design. Since the chrominance components carry very little spatial information in comparison to the luminance component, a process might focus resolution enhancement efforts on the luminance channel alone. Moreover, the computational demand of the multi-field enhancement system can be reduced by working with a coarser set of chrominance samples than that used for the luminance component.
A second observation concerning analog video is that the luminance component is often heavily aliased in the vertical direction, but much less so in the horizontal direction. This is to be expected, since the optical bandwidth is roughly the same in both the horizontal and vertical directions, but the vertical sampling density is less than half the horizontal sampling density. Moreover, newer video cameras employ CCD sensors with an increasing number of sensors per row, whereas the number of sensor rows is set by the NTSC standard. Empirical experiments confirm the expectation that high horizontal frequencies experience negligible aliasing, whereas high vertical frequencies are subjected to considerable aliasing. Hence, it is unlikely to be possible to increase the horizontal resolution of the final still image through multi-field processing; however, it should be possible to "unwrap" aliasing components to enhance the vertical resolution and remove the annoying aliasing artifacts ("jaggies") around non-vertical edges.
Hence, what is needed is a method and system for combining the information from multiple video fields into a single, high quality still image.